Time_lapse: immersive interaction with historic 3-D stereo images

Mankind’s fascination with three-dimensional images and stereo vision can be traced back to the ancient Greeks, and it was photography and the invention of the stereoscope during the mid-nineteenth century that helped to spur large-scale public interest in stereo images and stereo viewing techniques. Stereoscopes made it possible to see a pair of slightly different two-dimensional photographs in three dimensions by making use of the human eye’s parallax. With this century’s new possibilities in three-dimensional computing, historic stereoscopic images can now be used for the creation of interactive virtual environments. Driven by the wish to make historic stereo images interactively accessible to the user, we have developed an immersive virtual environment called "Time_lapse." This system allows two remotely located users to enter and interact with historic stereo images through full-body integration and immersion. 1. A Brief History of Stereoscopic Vision 1.1. Early forms of stereo vision The fascination with the concept of stereo vision goes back to ancient Greece around 300 B.C. Euclid for the first time explained the principle of binocular vision. He demonstrated that the right and left eyes see slightly different versions of the same scene and that the merging of these two images produces the perception of depth. In the sixteenth century, Leonardo Da Vinci experimented with perspective in an effort to create the impression of depth in his paintings. Around the same time, the Florentine painter Jacopo Chimenti created pairs of "stereo" drawings. During the Industrial Revolution, demand for more sophisticated forms of viewing resulted in the developments of new techniques, including magic laterns, the polyorama panoptique, peep shows and the kaleidoscope. [1] 1.2. Brewster’s Stereoscope However, it was the invention of photography that really made mass-culture 3-D viewing possible. The first patented stereo viewer was Sir Charles Wheatstone’s reflecting stereoscope in 1838. The device was a bulky and complicated contraption that utilized a system of mirrors to view a series of pairs of crude drawings. In 1844 a technique for taking stereoscopic photographs was demonstrated in Germany, and a much smaller and simpler viewer that utilized prismatic lenses was developed in Scotland by David Brewster (Fig. 1). Fig. 1 Stereo viewer for "Crystal Palace" Henry NEGRETTI & Joseph ZAMBRA. 1.3. Crystal Palace a breakthrough for the stereoscope The real breakthrough for Brewster’s Stereoscope came in 1851 with the opening of the Great Exhibition in London’s Crystal Palace located in Hyde Park. Many countries of the world were represented in an extravagant display housed in this huge glass building designed by engineer Joseph Paxton. Its beautiful domed roof (Fig. 2) made it the perfect setting for the stereo photographs taken by the company Negretti and Zambra. The various attractions in the Crystal Palace included "The Medieval Hall," "The Lotus Pond," "Egypt" (Fig. 3), and "Rome," among others. When Queen Victoria took a fancy to the Stereoscope at the Crystal Palace exposition in 1851, stereo viewing became vastly popular in Britain. [2] Fig. 2 "Crystal Palace" Henry NEGRETTI & Joseph ZAMBRA, 1851-52. Fig. 3 "Crystal Palace" "Egypt", Daguerreotype, Henry NEGRETTI & Joseph ZAMBRA, 1851-52. 1.4. Touring the world from home The stereoscope slides that were produced allowed people to sit in their own home and tour the world. The most popular slides were travelogues that showed the world from the abbeys and countrysides of Europe to the pyramids and tombs of ancient Egypt, to the Great Wall of China, and to the Taj Mahal (Fig. 4). The great events of the day found their way onto the stereo slides. The building of the Panama Canal, the terrors of war, and the destruction of natural disasters such as earthquakes were brought into homes in much the same way as television does today. By the 1870’s, local commercial photographers had sprung up around the country, and for a fee they would produce stereo slides of one’s farm, family or shop (Fig. 5). Fig. 4 "Taj Mahal" Stereo-photographic picture, International Stereoscopic Association, Tokyo Japan, 1908. Fig. 5 "Actors, Siam" Stereo-photographic picture, International Stereoscopic Association, Tokyo Japan, 1908. 2. "Time_lapse:" immersive interaction with 3-D stereo images Inspired by the collection of historic stereo images at the Tokyo Metropolitan Museum of Photography [3] and the wish to create an interactive virtual environment where users could interact with these historic images, we developed an immersive virtual environment called "Time_lapse." This system allows two remotely located users to enter and interact with the historic stereo images. To adapt the images for the construction of our three-dimensional immersive interactive environment and to provide the user with an immersive interaction experience, various image preparation processes were needed. These are described below. 2.1. Image preparation The following sections describe the image processing steps performed on the stereo images to make them suitable for interactive viewing and exploration by the user. There are three main processes: image acquisition, depth extraction and virtual views generation for motion parallax. The image’s depth is used to achieve the interposition depth clue, where near objects occlude far objects. Virtual views are synthetically generated to enhance the depth effect provided by the motion parallax clue, where near objects move more than distant ones. 2.1.1. Image acquisition A total of 15 historic colour stereo images were selected from the collection of the Tokyo Metropolitan Museum of Art. Left and right image pairs were scanned from a book[1] at 300 dpi and then scaled down to a 512x512 pixel resolution. No colour adjustment was made during the scanning process. Due perhaps to their antiquity, the images’ quality is anything but optimal. Judging from the era they were taken, we can conclude that they are indeed black and white images. However, some of them seem to be retouched to give them a coloured appearance. Also, the colours are fading away. These image sets could be viewed with a rather simple stereo-viewer provided with the book. The images were taken using stereo cameras with lenses separation, also called inter-ocular separation or baseline, that is roughly equal to the average human eye’s separation of 55-65 mm. Most of the scenes were carefully chosen to give a relatively good stereoscopic effect in spite of the short lens separation. 2.1.2. Depth extraction Due to the less than optimal quality of the scanned images, the image colours between left and right images differed significantly, so a general correlation-based stereo matching algorithm to extract a dense depth map was not appropriate. However, we applied a recursive and adaptive stereo matching algorithm [4], based on the Kanade-Okutomi[5] algorithm, to our images. No image rectification was needed since the camera lenses’ setup assures that epipolar lines coincide with raster scan lines in the image. Depth extraction results were, as expected, not very precise (Fig. 6).